Local Convergence of the Heavy-ball Method and iPiano for Non-convex Optimization
نویسنده
چکیده
A local convergence result for abstract descent methods is proved. The sequence of iterates is attracted by a local (or global) minimum, stays in its neighborhood and converges. This result allows algorithms to exploit local properties of the objective function: The gradient of the Moreau envelope of a prox-regular functions is locally Lipschitz continuous and expressible in terms of the proximal mapping. We apply these results to establish relations between an inertial forward–backward splitting method (iPiano) and inertial averaged/alternating proximal minimization.
منابع مشابه
iPiano: Inertial Proximal Algorithm for Nonconvex Optimization
In this paper we study an algorithm for solving a minimization problem composed of a differentiable (possibly nonconvex) and a convex (possibly nondifferentiable) function. The algorithm iPiano combines forward-backward splitting with an inertial force. It can be seen as a nonsmooth split version of the Heavy-ball method from Polyak. A rigorous analysis of the algorithm for the proposed class o...
متن کاملLong term motion analysis for object level grouping and nonsmooth optimization methods = Langzeitanalyse von Bewegungen zur objektorientierten Gruppierung und nichglatte Optimierungsmethoden
This work deals with theoretical and practical aspects of convex and nonconvex optimization algorithms for several classes of problems. They are applied to several (low-level) computer vision tasks. The optical flow motion estimation problem, which is among them, is a potential source for improving motion based segmentation methods. The second, more practical part of this work focuses on such a...
متن کاملIpiano: Inertial Proximal Algorithm for Non-convex Optimization
This paper studies the minimization of non-convex and non-smooth composite functions. In particular, we discuss the algorithm proposed by Ochs et al. in [14], called iPiano. Following [14], we present a global convergence result for functions satisfying the Kurdyka-Lojasiewicz property [12, 11] which is based on the work by Attouch et al. [4]. Furthermore, we discuss the implementation of iPian...
متن کاملAn Intelligent Approach Based on Meta-Heuristic Algorithm for Non-Convex Economic Dispatch
One of the significant strategies of the power systems is Economic Dispatch (ED) problem, which is defined as the optimal generation of power units to produce energy at the lowest cost by fulfilling the demand within several limits. The undeniable impacts of ramp rate limits, valve loading, prohibited operating zone, spinning reserve and multi-fuel option on the economic dispatch of practical p...
متن کاملLinearly convergent stochastic heavy ball method for minimizing generalization error
In this work we establish the first linear convergence result for the stochastic heavy ball method. The method performs SGD steps with a fixed stepsize, amended by a heavy ball momentum term. In the analysis, we focus on minimizing the expected loss and not on finite-sum minimization, which is typically a much harder problem. While in the analysis we constrain ourselves to quadratic loss, the o...
متن کامل